International Journal for Quality in Health Care 20 I 3; Volume 25, Number I : pp. I —7 
Advance Access Publication: 4 January 2013 



IO.I093/intqhc/mzs079 



Prospects for comparing European 
hospitals in terms of quality and safety: 
lessons from a comparative study in five 
countries 

SUSAN BURNETT 1 , ANNA RENZ 1 , SIRI WIIG 2 , ALEXANDRA FERNANDES 3 , ANNE MARIE WEGGELAAR 4 , 
JOHAN CALLTORP 5 , JANET E. ANDERSON 6 , GLENN ROBERT 7 , CHARLES VINCENT 8 AND 
NAOMI FULOP 9 

'Centre for Patient Safety and Service Quality (CPSSQ), Division of Surgery and Cancer, Department of Surgery, Faculty of Medicine, 
Imperial College London, Room 508 Medical School Building, St Mary's Campus, Norfolk Place, W2 IPG London, UK, Department of 
Health Studies, Faculty of Social Sciences, University of Stavanger, N-4036 Stavanger, Norway, 3 Department of Marketing, Operations and 
Management, ISCTE Lisbon University Institute, 1649-026 Lisbon, Portugal, Department of Health Policy and Management, Erasmus 
University, Campus Woudestein, PO Box 1738, 3000 DR Rotterdam, The Netherlands, The Jonkoping Academy for Improvement of 
Health and Welfare, School of Health Sciences, Jonkoping University, PO Box 1026, S-551 1 1 Jonkoping, Sweden, 6 Florence Nightingale 
School of Nursing and Midwifery, James Clerk Maxwell Building, 57 Waterloo Rd, SE1 8WA, London, UK, National Nursing Research 
Unit, King's College London, James Clerk Maxwell Building, 57 Waterloo Road, SE1 8WA London, UK, s Division of Surgery and Cancer, 
Department of Surgery, Faculty of Medicine, St Mary's Hospital, Imperial College London, 10th Floor QEQM Wing, W2 1NY London, 
UK, and 'Department of Applied Health Research, University College London, 1-19 Torrington Place, WC1E 7HB London, UK 

Address reprint requests to: Susan Burnett, Centre for Patient Safety and Service Quality, Faculty of Medicine, Imperial College London, 
Room 508 Medical School Building, St Mary's Campus, Norfolk Place, W2 IPG London, UK. Tel: +07802-755865; Fax: +0207+5943137; 
E-mail: s.burnett@imperial.ac.uk 

Accepted for publication 2 December 2012 



Abstract 

Purpose. Being able to compare hospitals in terms of quality and safety between countries is important for a number of 
reasons. For example, the 2011 European Union directive on patients' rights to cross-border health care places a requirement 
on all member states to provide patients with comparable information on health-care quality, so that they can make an 
informed choice. Here, we report on the feasibility of using common process and outcome indicators to compare hospitals 
for quality and safety in five countries (England, Portugal, The Netherlands, Sweden and Norway). 

Main Challenges Identified. The cross-country comparison identified the following seven challenges with respect to compar- 
ing the quality of hospitals across Europe: different indicators are collected in each country; different definitions of the same 
indicators are used; different mandatory versus voluntary data collection requirements are in place; different types of organiza- 
tions oversee data collection; different levels of aggregation of data exist (country, region and hospital); different levels of 
public access to data exist; and finally, hospital accreditation and licensing systems differ in each country. 

Conclusion. Our findings indicate that if patients and policymakers are to compare the quality and safety of hospitals across 
Europe, then further work is urgently needed to agree the way forward. Until then, patients will not be able to make 
informed choices about where they receive their health care in different countries, and some governments will remain in the 
dark about the quality and safety of care available to their citizens as compared to that available in neighbouring countries. 
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Introduction 

There are many reasons why it is important to be able to 
assess and compare hospitals in terms of quality and safety 
both within and between countries, for example to help 



hospitals improve their services and for governments to 
understand the impact of their policies. As it becomes more 
common for patients to seek health care in different coun- 
tries [1], the need for comparative information to enable 
choice is becoming more important. This has been 



International Journal for Quality in Health Care vol. 25 no. I 
© The Author 201 3. Published by Oxford University Press. 

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.Org/licenses/by-nc/3.0/), 
which permits non-commercial reuse, distribution, and reproduction in any medium, provided the original work is properly cited. I 



Burnett et al. 



recognized by the recent European Union (EU) directive on 
cross-border care [2], placing responsibility on countries to 
provide relevant information about the health care they 
provide to help patients make a choice based on the availabil- 
ity, safety and quality. 

Here, we report on the challenges identified in developing 
a framework of available indicators and information that 
could be used to assess hospitals based on the quality and 
safety in five European countries, considering the issue 
involved and making recommendations for the future. The 
countries are England, Portugal, The Netherlands, Sweden 
and Norway; countries representing variation in important 
aspects of health care, for example, funding arrangements 
and health-care quality [3]. Health services research teams 
working in leading universities in these countries are partici- 
pating in the EU-funded Quality and Safety in European 
Hospitals research study [4, 5], a study in 10 hospitals, 2 in 
each country [6] exploring the organizational and contextual 
factors affecting how hospitals implement quality improve- 
ment. The purpose of the selection process for the study 
was to find hospitals at different stages of the quality 
journey, rather than only those seen to be doing well. 



Indicators available in each country to 
assess and compare hospitals 

Ten commonly used quality and safety process and outcome 
indicators were proposed to identify hospitals with different 
levels of performance in each country, indicators that are 
widely regarded in the medical field as good practice 
(Table 1), including operating on hip fractures within 48 h. 
The research teams were asked whether the indicators were 
available; who had access; were they available for every hos- 
pital; how robust were the data (including definitions); and 
what other indicators were available? 

Table 1 summarizes the responses received, with only 
three outcome indicators available in all countries: infection 
rates; a range of condition-specific mortality rates; and 
Caesarean section rates and only one process indicator: hip 
fractures treated in a set period. In general, these indicators 
were publicly available, but in Portugal, this was more com- 
plicated as described later in this paper, and in all cases, the 
researchers had to know where to look for the information 
from different organizations and websites. 

The data for methicillin-resistant Staphylococcus aureus 
(MRSA) bacteraemia were not considered to be helpful in 
the Netherlands and Sweden, where the rates are very low as 
compared to the UK, where this indicator is considered very 
relevant and where until recently, the numbers have been 
much higher. It was apparent that although available in all 
countries, these indicators were not directly comparable 
because of differences in definitions (Table 2). For example, 
the surgical site infection rates in some countries are based 
on 1 day prevalence surveys, whereas in others, surveillance 
systems operate, which include every patient, and in each 
country, different surgical procedures are included, some are 



mandatory, whereas others are voluntary. The indicator of 
hip fracture treatment ranged from operating within 18 h in 
Sweden to 48 h in England and was only available for people 
aged over 65 in Norway. 

The following sections provide more detail by country. 

England 

In England, data are collected centrally from each hospital 
organization through the Hospital Episode Statistics data- 
base. These data are analysed by Dr Foster Intelligence (a 
partnership between the UK Department of Health and a 
commercial provider of health-care information), and each 
year, a hospital guide is made available to the public, setting 
out the position of each hospital against a range of indicators 
[7]. Data on every patient with a blood stream MRSA infec- 
tion are collected with aggregated figures for each hospital 
made available on a public website [8]. 

Portugal 

Since 2001, the Health Minister has provided the 
Organisation for Economic Co-operation and Development 
(OECD) with process and outcome indicators similar to 
those in Table 1. However, it is not possible to disaggregate 
the data into regions and hospitals, making individual hos- 
pital comparisons impossible. In the last decade, various 
reforms, such as the merger of hospitals into clusters and 
changes in the mix of public— private ownership, have led to 
past performance data being lost in the merged groups, 
and some are only now producing group wide data. 
Furthermore, hospital organizations have autonomy to 
choose the processes and outcomes indicators that they use, 
and these are their exclusive property. In this context, access 
to good quality performance data was almost impossible in 
Portugal, and the indicators included in the Performance 
Evaluation Reports of Public Portuguese Hospitals [9] were 
used as a proxy. These reports of raw data have been pub- 
lished annually since 2005 by the National School of Public 
Health [10], but are not adjusted for age or case mix. 

Netherlands 

In the Netherlands, there was a wealth of publicly available 
data about quality in hospitals, predominantly derived from 
the information of the Dutch Health Care Inspectorate 
(IGZ) [11] and listed by a newspaper in an annual top 100 
hospitals [12]. This is supported by further information of 
the Dutch Association of Hospitals (NVZ) [13] such as hos- 
pital mortality rates. Infection data are separately collected 
for all hospitals, but presented in a combined way on a 
health-care comparison website [14]. Process measures were 
chosen from a wide range of publicly available information, 
including pressure ulcers and eye examination for diabetes, 
measures collected by the hospitals, but not externally 
validated. 
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Table I Quality and safety indicators available in each country highlighting difference between countries 



Indicator 


England 


Portugal a Netherlands 


Sweden 


Norway 


Indicators 
available in 
all countries 


C-diff or MRSA 


Yes, 


Yes 


Data are collected at a 


Yes, but data are 


Data are not 




rates 


available 




national level from all 


not considered 


available at the 






for every 




hospitals. Low rates, so 


useful to compare 


hospital level. 






hospital 




information is not used 


hospitals because 












to compare quality in 


of the low rates 












hospitals 








Surgical site 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


infection rates 














Composite 


Yes 


Yes 


Yes 


For some hospitals, 


Not available in time 




mortality rate 


HSMR 




Potentially avoidable 


not all 


for this process — 










mortality rate in 




now available 










proportion of actual 














mortality rate of the 














hospital 








Specific mortality 


Yes 


1 LS 


ICS 


Yes 


Some, but not all 


Yes 


rates (AMI, 










available in time for 




stroke, CABG 










this process — all 




and AAA) 










now available 




Emergency 


Within a 


Yes 


Only for heart failure 


Some diagnoses, 


Not available 




readmission rates 


set period 




readmissions within 12 


not all 


publicly (could be 










weeks 




requested from the 














Norwegian Patient 














Register for a fee) 




Third and fourth 


Yes 


1NO 


Yes, but not available at 


Yes 


Not available in time 




degree perineal 






the hospital level for 




for this process — 




trauma rate 






comparison (voluntary 




now available 










data collection by 














obstetricians) 








Caesarean 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


section rate 














Primary 


Yes 


Yes 


Yes 


Yes 


No, but could be 




angioplasty rates 










requested from the 














Norwegian Patient 














Register on payment 














of a fee (see above) 




Hip fractures 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


treated in set 














time 














24 h brain scan 


Yes 


Yes 


No 


No 


No 




rate for stroke 















AMI, acute myocardial infarction; AAA, abdominal aortic aneurysm; CABG, coronary artery bypass graft; HSMR, hospital standardized 
mortality indicator. 

Tn Portugal, the information was available but was not available publicly by hospital. 



Sweden 

Sweden has a tradition of collecting data nationally; however, 
more quality data have been collected over time by county 
councils (who manage hospitals in Sweden) and private health 
organizations. There are also many disease-specific quality 
registries. One of the key issues in Sweden is that most data so 
far have been published at a county council level, but there 



were enough data published at a hospital level to satisfy the 
needs of the project. An increasing number of hospital-specific 
quality indicators are now being published, and a process is 
ongoing between the Swedish Association of County Councils 
and Regions [15] and the National Board of Health and 
Welfare [1 6] to analyse and make available national information 
in an accessible way and for comparative purposes. 
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Table 2 Examples of differences in the definition of indicators between countries 



Indicator 



Country Definition 



Surgical site 
infections 



Caesarean 
section rate 



Hip fractures 
treated within a 
set period 



24 h brain scan 
rate for stroke 



England Surveillance methodology used. Hospitals are able to choose from 14 categories of surgical 
procedures. Data are collected on each patient who has a procedure in the category under 
surveillance. SSIs that meet standard case definitions are identified through active follow-up 
during the post-operative stay or through readmission 
Netherlands Percentage of patients with post-operative site infection within 30 days after surgery with at 
least one of the following symptoms: pain, redness, local swelling and heat, (based on the 
definitions of European Centre for Disease Prevention and Control) 
Norway The number of demonstrated hospital infections — of the four most common, urinary tract, 
lower respiratory, post-operative wound and septicaemia — at a given point of time, in relation 
to the total number of hospitalized patients at the same time. Post-operative wound infection 
data collected for: (i) heart surgery, (ii) Caesarean section, (iii) insertion of prosthesis in the hip 
joint, (iv) removal of gall bladder and (v) surgical intervention on colon 
The proportion of all deliveries that were a Caesarean section 



England, 
Netherlands 
and Norway 
Sweden 
England 

Netherlands 

Sweden 
Norway 

England 

Netherlands 



Percentage of Caesarean sections at uncomplicated delivery, age 20—34 years 

The percentage of all patients with a fracture neck of femur as primary diagnoses that have a 

related procedure within 2 days. 

Percentage of patients with ASA 1 & 2 classifications operated within 1 day. Also, percentage 

of patients with ASA 3 & 4 classifications operated within 1 day 

Percentage of hip fractures treated within 18 h of arrival at hospital 

Proportion of patients over 65 years with hip fracture, treated within 48 h after being 

hospitalized with needs of immediate help 

Proportion of stroke patients who have a brain scan (CT or MRI) performed on the day of 
admission and within 1 day of admission 

Average time between the admission by TIA or Stroke and the completion of standard 
diagnostic tests — blood tests, brain imaging, ECG and imaging of the neck vessels. 



ASA, American society of anesthesiologists; SSIs, surgical site infections; TIA, trans ischaemic attack. 



Norway 

In Norway, hospital indicator data are reported by the hospi- 
tals to the Norwegian Patient Register and the Norwegian 
Institute of Public Health and published on a website called 
'Free Hospital Choice' [17]. The Norwegian Directorate of 
Health [18] publishes data on a growing number of national 
quality indicators in acute care. It is mandatory in Norway to 
report MRSA infections, but this information is not available 
for each hospital. The process indicators available were dif- 
ferent here to other countries and included the number of 
patients placed on a bed in a corridor, bathroom, laundry 
room or living room at 7 a.m. and the proportion of patient 
discharge letters transferred within 7 days to the general 
practitioner. 

Hospital accreditation or licensing 
schemes in each country 

Table 3 describes the hospital licensing, accreditation or 
similar schemes in each country showing that this type of 



information is not available to be used to assess or compare 
the quality of hospital care in Europe. 

Only in England is there is a statutory licensing scheme 
for all hospitals [19]. In the other countries, accreditation or 
certification is voluntary, and quality award schemes are often 
based on the Baldridge Award [20] . 

Challenges and potential solutions 

These findings highlight the differences between countries in 
the data and information available about the quality and safety 
of hospital care. In England, the Netherlands and Norway, 
various hospital data are submitted to national organizations 
that conduct analysis and make them available to the public by 
hospitals. In Sweden, the level of aggregation of the data is 
stronger at the county council level and in the disease-specific 
quality registries. In Portugal, the information that is collected 
at the national level is not easy to obtain by region or hospital. 
In all countries, data on specific procedures or conditions are 
collected by organizations, including medical societies, but not 
all make this available to the public. 
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Table 3 Accreditation or licensing systems for hospitals in 
each country 

Country Accreditation/licensing schemes for hospitals 

England Care Quality Commission licensing scheme 

[1 9] statutory requirement with 'Annual Health 
Check' of all hospitals producing publicly 
reported quality ratings for each hospital 
organization [27]. 

Portugal No national accreditation scheme, but one is 
in the validation phase — some hospitals 
voluntarily accredited with Joint Commission 
[28] and the Health Quality Service (formerly 
Kings' Fund) [29] 

Netherlands Voluntary accreditation, so not all hospitals are 
involved. Accreditation undertaken by the 
Netherlands Institute for Accreditation in 
Healthcare [30] based on the Baldridge Award 
[20]. Some hospitals use other accreditation 
systems, i.e. ISO and INK (Institute for 
Dutch Quality). 

Sweden No formal hospital accreditation system. 

'Baldridge' type award scheme in operation 
[20] 

Norway No formal hospital accreditation system. A 

small number of institutions providing clinical 
services are certified according to ISO 9000 
by accredited certification bodies. 

ISO, International Organization for Standardization; INK, Instituut 
Nederlandse Kwaliteit. 



The differences in the indicators collected by countries 
highlight the different policy concerns of those countries. In 
England, for example, data are available on MRSA infections 
for each hospital, whereas in Norway, no MRSA hospital 
level data are available, but data are available on the propor- 
tion of patients placed in a bed in a corridor or living room 
at 7 a.m. Each country also has a different approach to the 
assessment of hospitals in terms of accreditation or licensing 
with most having voluntary schemes. Only England has 
statutory licensing and an 'annual health check' of hospitals. 



In recognition of the requirement for all citizens to have 
access to high-quality, safe health care, the EU has been 
engaged in a number of projects to support hospitals in this 
quest and to find ways of monitoring and measuring this. 
These include the SIMPATIE project [22] that reported in 
2007 on the development of indicators and outcome mea- 
sures and a vocabulary for patient safety across Europe. The 
OECD [23] and WHO [24] have also funded projects to 
develop indicators for hospital quality and safety. However, 
the findings here show that despite all this important work, 
there are very little data available on the quality and safety of 
hospital care that is comparable between these five countries 
in Europe, and we anticipate that this is likely to be the case 
across the EU and in countries outside Europe, where citi- 
zens also travel for health care. 

In 2008, the European Observatory on Health Systems 
and Policies [25] reported that the EU needs to answer the 
question of how citizens of Europe can be assured that they 
will receive high-quality care if they need health care beyond 
their national frontiers. Both OECD [26] and the 
SIMPATIE [22] project have recommended that the quality 
of administrative data for health services in the EU needed 
to improve. Our study indicates that to date these issues have 
not been addressed and that further work is urgently needed 
to find a way forward if patients are to have any information 
to make choices between health-care providers when they 
travel for treatment in Europe. Clearly, the solution will need 
to be of low cost with the information easily kept up-to-date. 
The first step may be for those countries without compara- 
tive data about the safety and quality of their hospitals to be 
required to assemble available information from hospitals 
within their borders, to analyse this information and place it 
in the public domain. In the longer term to provide informa- 
tion for comparison between countries, the solution may be 
to agree to a small set of well-defined indicators to be col- 
lected by each hospital organization in Europe with some 
common standards and methods for assessing these. 
Alternatively, the solution may be a patient-led hospital 
equivalent of an internet hotel comparison website such as 
'Trip Advisor' (www.tripadvisor.com). In the meantime, 
patients travelling across the borders for their health care in 
Europe will not be able to make informed choices about 
where they receive their health care on any other basis than 
access, and many governments in Europe will remain in the 
dark about the quality and safety of care currently available 
to their citizens in their hospitals as compared to that 
available elsewhere. 



Discussion/conclusions 



The increasing movement of people across borders for 
health care, whether by choice or circumstance, is driving the 
need for more transparency about the quality and safety of 
health care in European hospitals. The new EU directive on 
patients' rights [2] places a responsibility on member states to 
'provide relevant information to help individual patients to 
make an informed choice, including on treatment options, 
on the availability quality and safety of the health care they 
provide'[21]. 
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